Multi-measurement method of comparing and normalizing assays

ABSTRACT

A computer-implemented statistical technique is provided for normalizing the response curves of multiple measurement methods. The end result is a group of response curves, one for each measurement method under consideration, which depend on a common independent variable—the actual physical property being measured by the methods. The results are provided as a collection of equations, curves, and/or tables (a “nomogram”) to facilitate conversion of measured values from one method to measured values from a second method. In the technique, data is provided for each of the methods being normalized. The input data includes measured values from common samples which are analyzed by two or more of the methods under consideration. The technique also requires assumptions or approximations of the true physical property values for each of the samples used to generate the data. Still further, the technique requires assumptions of the mathematical form of the response curves (e.g., linear, sigmoidal, etc.). The computer system then solves all parameters appearing in the mathematical expressions simultaneously. At the same time, the computer system solves for one or more “correction factors” to some of the approximations for the true physical property values of the samples. This specifies the relative separation distances between the physical property values on the independent variable axis.

BACKGROUND OF THE INVENTION

The present invention is directed to computer implemented statisticalmethods for comparing various assays or other measurement methods andconverting between their results. More specifically, the presentinvention provides computer implemented statistical methods fordetermining the response curves of multiple measurement methods asdefined for a single independent variable (which is the underlyingproperty being quantitated by the methods).

Many assays and related tests quantify a physical property of a sampleby comparing a measured assay value against an assay curve for thephysical property of interest. For example, a blood sample may containsome initially unknown level of a particular pathogen. When the sampleis evaluated with an assay for the pathogen of interest, it provides ameasurable signal which tends to be proportional to the pathogen levelin the patient's blood (at least in a log—log representation).

Examples of measured signals include luminescence or radiation emittedfrom a test sample, the absorption coefficient of the sample, the colorof a sample, etc. In a typical case, the assay procedure involvescontacting a test sample (analyte) with a test solution followed by awashing step. Thereafter, the test quantity of interest is measured andcompared against an assay curve (sometimes referred to as a “responsecurve”). The assay curve provides the measured value as a dependentvariable and the “true value” of the property of interest as anindependent variable. In one specific example, an assayed sample ofhepatitis B virus (HBV) DNA emits light of a luminescence that varieswith viral load. Thus, the luminescence of the sample is detected andcompared against the assay curve which specifies a corresponding valueof viral load for the sample.

In most useful assays, the assay curve increases monotonically with theproperty of interest over the dynamic range (e.g., luminescenceincreases monotonically with viral load). Often the assay is designed sothat the response of the assay is nearly linear over a specific dynamicrange. To achieve this, the assay curve may be expressed as thelogarithm of the measured value versus the logarithm of the propertyvalue. In practice, however, such assay curves rarely assume a trulylinear form. Frequently, there is a slight curvature over the dynamicrange which can be better represented by a quadratic expression.Further, near or just beyond the limits of the specific dynamic range,the response curve often flattens (i.e., the measured value changes onlyslightly with respect to changes in the true property value) to give theoverall response curve a “sigmoid” shape.

Even with widely used and validated assays, one is never certain thatthe specified property value for a sample is truly accurate. Forexample, the calibration of an assay may be inaccurate because the“standard” used to generate the assay curve is itself inaccurate.Sometimes, the property value of a standard changes slightly with time.And sometimes when a standard runs out, the new standard created toreplace the old does not possess the same true property value as the oldstandard. Further, while a given assay may be internally consistent overa period of time, it is still very difficult or impossible to accuratelycorrelate two different assays for the same analyte.

Many applications could benefit from improved confidence in measurementsof the property value consistent across assays. For example, one mightwant to use two or more different assays to monitor the same propertyvalue. A hepatitis B patient may have had his viral load monitored witha first assay that becomes temporarily unavailable. When a secondassay—which relies on an entirely different physical mechanism than thefirst assay—is used in place of the first and gives a rather highreading of viral load, it could mean either (a) the patient's viral loadis truly increasing or (b) the second assay employs an assay curve that,in comparison to the first assay's curve, gives a higher property valuereading for a given sample. Obviously, an attending health careprofessional needs a reliable value consistent with both assays.

Also, parametric models for predicting the outcome of a medicaltreatment or other course of action are created from prognosticvariables relevant to the models (e.g., assay results). The accuracy ofthe model is improved as more-consistent data is used to construct it.If that data is provided as assay results for two or more assays, theremust be some way to establish a conversion between the assay results ofthe two or more assays. Otherwise the resulting model may fail toaccurately handle inputs from one or more of the assays used toconstruct the model.

Other applications exist that require a conversion between propertyvalues specified by multiple assays or methods. For example, when anenterprise generates a new assay standard it must accurately correlatethat standard's property value to the old standard's property value.Otherwise, assays using the new standard will not be consistent with thesame assays using the old standard.

In another example, enterprises may need to compare two assays'performance (e.g., sensitivity and responsiveness) when those assays aredesigned to quantitate the same analyte. Several commercial assays areavailable for HBV DNA quantification, and laboratory managers need toolsto assess how well the various assays operate. Even when results arereported in the same unit of quantification for a given sample,different assays report different results. Thus, the person conductingthe comparison must ensure that the response curves of the two assayscan be plotted on the same independent variable axis (the true propertyvalue axis).

Traditionally, when comparing multiple assays or batches of a standard,one uses a regression analysis to quantify the associations of interest.For example, for a series of samples, the measured values of a firstassay or batch is provided as the independent variable and the measuredvalues of the second assay or batch is provided as the dependentvariable. Then one assumes a relationship between the independent anddependent variables (e.g., a linear or quadratic relationship) and aregression analysis is performed to identify parameters of therelationship that nicely fit the data. Unfortunately, linear regressionanalysis is restricted to comparison of only two assays at a time. Stillfurther, this application of regression tends to violate a primaryassumption for correct inference of results, i.e., the independentvariable is assumed to be more precise versus the dependent variable.See e.g., Yonathan Bard, “Nonlinear Parametric Estimation,” AcademicPress, New York, N.Y., 1974. Hence, the more precise variable must bethe independent regression variable, typically denoted as x, while thenoisier variable must be the dependent or response variable, denoted y.In many assay comparisons, where one assay is selected to be y variableand the other x variable, the results are questionable since the assayerrors are comparable or, worse, the x variable error is larger than they variable error.

Hence, there is a need to compare multiple assays (or other methods) orbatches of standard without the inherent bias of linear regression, tobe able to convert values between the different assays or standards, andto provide a property basis consistent across all assays.

SUMMARY OF THE INVENTION

The present invention provides computer implemented statisticaltechniques for normalizing the response curves of multiple measurementmethods. This permits, inter alia, evaluation of multiple methods,conversion of their results, and normalization of assay standards. Thesetechniques allow comparative assessment of two or more measurementmethods at one time and significantly reduce the bias as compared to thetypical regression methods. The end result is a group of responsecurves, one for each measurement method under consideration, whichdepend on a common independent variable—the actual physical propertybeing measured by the methods. The response curves may be provided as acollection of equations for the curves, plots of the curves, and/ortables (collectively a “nomogram”) to facilitate conversion of measuredvalues from one method to expected measured values of any other method.

In the technique, the computer receives data for each of the methodsbeing normalized. The data includes measured values from common sampleswhich are analyzed by two or more of the methods under consideration.The technique also requires assignments or approximations of the truephysical property values for each of the samples used to generate thedata (e.g., sample 1 has concentration of 1M, sample 2 has aconcentration of 2M, etc.). Still further, the technique requiresassumptions of consistency within each assay expressed in mathematicalform as the response curves (e.g., the response of assay 1 is linear,the response of assay 2 is sigmoidal, etc.). The computer system thensolves all parameters appearing in the mathematical expressionssimultaneously (e.g., it solves for the slope and intercept of a linearresponse curve). Simultaneously, the computer system solves for one ormore “correction factors” to some of the approximations for the truephysical property values of the samples. When solved, these correctionfactors correct the relative separation distances between the initialapproximations of physical property values on the independent variableaxis, thereby reducing errors in x-axis values and improvingself-consistency across assays. In the end, each response curve isprovided on a common x-axis, thereby allowing direct comparison andconversion.

In one aspect, the computer implemented methods of this invention may becharacterized by the following sequence: (a) receiving data specifyingmeasured values versus physical properties or samples evaluated by thetwo or more measurement methods; (b) assuming a value for the physicalproperty of a first sample (tested by at least two of the methods) andassuming a value for the physical property of a second sample (alsotested by at least two of the methods); (c) for each of the two or moremeasurement methods, assuming a mathematical form of a response curve;and

(d) simultaneously solving for all unknown parameter values of theresponse curves and for a correction factor for the value of thephysical property of the second sample. The input data should include atleast enough data points to fully determine the parameter values of theresponse curve, the correction factor for the second sample, and anyadditional correction factors for other physical property values(associated with additional samples that were analyzed by two or more ofthe methods).

The mathematical form of the response curve may be any monotonicnon-constant function. Examples include linear expressions, quadraticexpressions, and sigmoidal expressions. Examples of unknown parametersvalues solved for with this invention include the slope and intercept ofa line, the curvature of a quadratic expression, etc. The computerimplemented method may output a nomogram including solved responsecurves or solved mathematical expressions for at least two of themeasurement methods.

The method is invariant (indeterminant) with respect to translations onthe x-axis. Therefore a constraint is added to enable a solution to theequations. One such constraint (but not the only possible constraint) isan “anchoring procedure.” Typically, when the method simultaneouslysolves for the parameter values, it will solve for multiple correctionfactors, each associated with the physical quantity of a sample forwhich data was received. Prior to solving the expressions, however, oneof the physical property values is set as an “anchor value.” Duringsimultaneously solving for all the unknown parameter values, the anchorvalue is not provided with a correction value.

These and other features and advantages of the present invention will bepresented in more detail in the following description of the inventionand the associated drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a graph of hypothetical data indicating the response of threeseparate measurement methods and application of one method of thisinvention.

FIG. 2 is a block diagram of a generic computer system useful forimplementing the present invention.

FIG. 3 is a process flow diagram depicting some of the important stepsemployed in a preferred computer implement technique of this invention.

FIG. 4 is a graph of hypothetical data illustrating some of the stepspresented in FIG. 3.

FIG. 5 is a nomogram showing response curves for three different HBV-DNAassays as generated by the techniques of this invention. The responsecurves were fit to quadratic expressions.

FIG. 6 presents the mathematical expressions for the three responsecurves shown in FIG. 5.

FIG. 7A is nomogram showing response curves for three different HBV-DNAassays as generated by the techniques of this invention. The responsecurves were fit to sigmoidal expressions.

FIG. 7B is a plot illustrating the features of a sigmoidal responsecurve as used to generate the nomogram presented in FIG. 7A.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A. OVERVIEW

The present invention provides techniques and systems for normalizingmeasurement results such that data taken by multiple measurement methodsis provided on a common independent variable axis. Thismulti-measurement technique is a minimal bias, non-linear regressiontechnique that can evaluate the performance characteristics of multiplemethods and produce a conversion between the readings of the multiplemethods. The invention accomplishes this by simultaneously solving forparameters in expressions for response curves and correction factorsapplied to the data values of the independent variables.

In the following description, various specific details are set forth inorder to fully illustrate preferred embodiments for accomplishing theanalysis of this invention. For example, certain specific applicationsof the invention (e.g., comparing multiple assays for HBV DNA) will bedescribed. It will be apparent, however, that the invention may bepracticed without limitation to the specified details and applicationspresented herein. Based on principles similar to those that govem leastsquares analysis, neural networks, latent analysis, and principlecomponent analysis, the multi-measurement technique of this inventionassumes that all measurement techniques being compared are measuring thesame unknown physical property (e.g., analyte concentration in the caseof assays). The multi-measurement technique fits the multiple data setsfrom different assays to determine the optimal interrelationshipsbetween the data sets such that a consistent “true” physical property isderived. These “true” values represent an intrinsic component of what isbeing measured by each assay (e.g., HBV DNA concentration), and servesas the unifying factor for all assays being compared. The measured assayvalues (typically provided as natural logarithms) for each specimen aredesignated as dependent (y) variables while approximate “true” valuesare designated as independent (x) variables. Approximation of the “true”physical property (x) in each specimen may be performed in a two-stageprocess. First, an initial approximation of analyte concentration (e.g.,viral load), designated x′, is obtained by normalizing the input values(usually via natural logarithm values) for each specimen tested byassays being compared, and averaging the log values for each specimen.The method may, during the averaging, weight each measurement by thesensitivity of the corresponding assay; i.e., results from steeper slopeassays get higher weights.

Then, the calibration function (expression for the response curve) andcorrections to analyte concentration approximations, designated δ, areestimated simultaneously for all specimens by non-linear regression. Theapproximated “true” analyte concentration (x value) in each specimen iscalculated using the equation: x=x′+δ. Once x′ is corrected to x, it isused to generate a multi-measurement technique nomogram.

The response curves on this nomogram may be plotted on a log transformedscale, where the y-axis indicates the measurement levels for thespecimens measured by each of the assays (in the various log assayunits), and the x-axis indicates the corrected “true” analyte logconcentration in each specimen as generated by the multi-measurementtechnique. Features of the assay curves, including curvature, slope, andthe degree of scatter about the curve, differ for each assay and providecomparative information about each assay's performance characteristicsand allow direct comparisons at each location on the x-axis.

The multi-measurement technique of this invention enhances the value ofmeasurement method quantification in several ways. First, themulti-measurement technique allows performance characteristic comparisonand interassay value conversion among multiple assays simultaneously.Second, it assesses and optimizes analytical standards to facilitatedevelopment of more precise and reliable assays. Third, normalization ofassay values allows researchers, scientists, physicians, or otherprofessional using measurement techniques to make sense of resultsreported from different assays with different units and standardization.This is because the response curves and equations generated by themulti-measurement technique make it possible to convert amongmeasurements of each method.

As mentioned, conventional assay evaluations using linear regressionanalysis promote internal bias because the assay depicted on the x-axis(the independent variable) is assumed to be significantly more precisethan the assay depicted on the y-axis (the dependent variable). Unlikelinear regression, the multi-measurement technique simultaneouslyestimates the true value of the analyte and the curves that associateeach assay's results with this value by assuming that all measuredvalues (e.g., luminescence) for each measurement method are dependentvariables A new more precise independent variable is constructed thatrepresents an intrinsic quality of what is being measured by all theassays, such as the viral load and relates each assay to the others.

To briefly illustrate techniques of this invention, FIG. 1 presents agraph of hypothetical data indicating the response of three separatemeasurement methods. The y-axis (vertical axis) represents the value ofthe measured quantities for each of the measurement methods. Becausethese methods may employ different measurement quantities (e.g.,luminescence in one case and absorptivity in another case andradioactivity in yet another case), the y-axis normally encompassesmultiple units which have an arbitrary relation to one another.Typically the relation is chosen so that all response curves can beviewed together (e.g., log measurement). The x-axis (the horizontalaxis) represents the value of the underlying physical property which isbeing measured. This axis will, of course, present only a single type ofunits (e.g., concentration of viral DNA) and will be consistent frommethod to method. For many applications, the absolute magnitudes of theunits on the x-axis need not be known because the techniques of thisinvention are merely making the multiple measurement methods internallyconsistent with respect to the underlying physical property. Thus, inmany embodiments, an “anchor” value is arbitrarily chosen on the x-axisand all other values on the x-axis are chosen and corrected relative tothe anchor.

In the hypothetical example of FIG. 1, three independent clinicalsamples are provided for analysis. They may be blood samples of patientsinfected with HBV for example. These samples have unknown values of theunderlying physical properties which can be approximated by a variety oftechniques. In this case, the initial approximations were 0.81 for afirst sample, 1.0 for a second sample, and 1.15 for a third sample. Inaddition, two dilutions of the first sample are made. In the firstdilution, the sample was diluted by 50% to approximately 0.5. In thesecond dilution, the first dilution was further diluted by 50% toapproximately 0.25. These approximate sample (and dilution) values arepresented on the x-axis.

Each of the samples was tested by two or more of the three measurementmethods. The data points for a measurement method denoted “A” are shownas “X”s, the data points for a second measurement method denoted “B” areshown as triangles, and the data points for a third measurement methoddenoted “C” are shown as round dots. The first sample was tested by allthree methods as indicated by data points for x=0.81 on response curvesof all three measurement methods. The second sample was also tested byall three methods (data points for x=1.0). The first dilution of thesecond sample was tested by the first and third (A and C) measurementmethods, and the second dilution of the second sample was tested by thesecond and third (B and C) measurement methods. Finally, the thirdsample (x=1.15) was tested by the first and second methods (A and B).

The data points shown in FIG. 1 may represent the processed result ofmultiple data points taken with a given method using a specified sample.To improve the accuracy of the inventive technique, multiplemeasurements (“replicates”) may be taken for some or all of themethod/sample combinations. All measurements show some randomness orscatter when taken multiple times. Typically the scatter may bereflected as a Gaussian or normal distribution having a mean andstandard deviation. In a preferred embodiment, all individual datapoints are input —without assuming the form of a noise distibution(Gaussian, log normal, etc.). The method then determines the best fitcurve passing a minimized distance from each mean.

In this example, the second sample was chosen as the anchor value andgiven a value of 1.0 on the x-axis. Its first and second dilutions weretherefore given values of 0.5 and 0.25 respectively. As mentioned, the xvalues of the first and third samples were approximated to be 0.81 and1.15, respectively. Except for the anchor value, all sample and dilutionvalues were provided with a correction factor, δ. The values of thesecorrection factors are solved by the computer techniques of thisinvention to improve the accuracy of the relative positions of thesamples on the x-axis. Simultaneously with the solution of the x-axiscorrection factors, the technique solves the various parameters used inthe expressions for the response curves. If a curve is assumed to bequadratic, for example, the technique must solve for the intercept, b,the linear component of slope, m, and the curvature, k. The relevantexpression is y=kx²+mx+b.

In one particular example, the response curves of all three measurementtechniques are assumed to be quadratic in order to account for somecurvature over their dynamic ranges. The computer system receives thedata points shown in FIG. 1 (which may actually represent the processedreadings of multiple readings taken for each sample/method combination).It assumes the quadratic forms of the response curves and the initialapproximations of the underlying property values (x-axis positions) ofthe samples. This means that there are several unknowns to be solved.These are (1) a correction factor, δ, for each x value of the sampleexcept the anchor value and (2) an intercept, b, a slope, m, and acurvature coefficient, k, for each response curve.

Specifically, the parameters to be solved are k_(A), m_(A), b_(A),k_(B), m_(B), b_(B), k_(C), m_(C), b_(C), δ₁, δ₂, δ₃, and δ₄. Theseparameters may be solved simultaneously by a computer systemimplementing a non-regression technique, for example.

Preferred embodiments of this invention make certain assumptions aboutthe methods being analyzed and the data input to the techniques of thisinvention. That is, certain criteria must be met in order for thenormalizing procedures of this invention to work. These assumptions arethe following.

First, multiple measurement methods for assessing the same physicalproperty must be compared together. The techniques of this invention donot act on just one method. Obviously, two mechanistically differentassays for quantifying HBV DNA meet this criterion. However, the rangeof methods that qualify is broader than this. For example, because agiven assay may behave slightly differently when conducted underslightly different conditions, a single assay when conducted under theseslightly different conditions may form the basis of two or moreindependent methods which can be employed with the multi-measurementtechnique of this invention. For example, one company's HBV DNA assaymay be performed with two different standards or on two different assayplates to provide two distinct methods as required to meet thiscriterion.

Second, samples to be corrected in the multiple measurement techniquemust be measured by at least two of the measurement methods underconsideration. Thus, a first sample will be measured with two or more ofthe methods, a different sample will be measured with two or more of themethods (not necessarily the same methods used with the first sample),etc. Of course, each different sample must contain the same analyte ofinterest—albeit at different levels—so that all methods measure theequivalent property.

Third, for each method, the number and distribution of data points(measured values for samples under consideration) is adequate to fullydetermine each of the parameters required to specify the assumed form ofthe response curves (e.g., two unique x points for a line (slope andintercept), three unique x points for a quadratic curve, etc.). Theminimum total of all properly distributed data required to handle allmeasurement methods in the normalization is the sum of the number ofparameters for each method curve plus the number of samples to becorrected used to generate the data plus one (for an anchor x value).This additional requirement is imposed by correction factors for allsamples except one (designated as the anchor value). As explained, thesecorrection factors are provided to allow adjustment of the relativeposition of the sample values but they are additional parameters thatmust be solved for.

Fourth, each individual measurement method has the property ofreasonable continuity or smoothness as the hidden phenomena varies. Inaddition, the response curves must be monotonic and non-constant. Thus,it can not have two values of the underlying property for any onemeasured value (i.e., a one-to-one mapping between measured value andunderlying property is required).

B. COMPUTER SYSTEMS FOR IMPLEMENTING THE INVENTION

Embodiments of the present invention as described herein employ variousprocess steps involving data stored in or transferred through computersystems. The manipulations performed in implementing this invention areoften referred to in terms such as calculating, normalizing, or solving.Any such terms describing the operation of this invention are machineoperations. Useful machines for performing the operations of embodimentsof the present invention include general or special purpose digitalcomputers or other similar devices. In all cases, there is a distinctionbetween the method of operations in operating a computer and the methodof computation itself. Embodiments of the present invention relate tomethod steps for operating a computer in processing electrical or otherphysical signals to generate other desired physical signals.

Embodiments of the present invention also relate to an apparatus forperforming these operations. This apparatus may be specially constructedfor the required purposes, or it may be a general purpose computerselectively activated or reconfigured by a computer program and/or datastructure stored in the computer. The processes presented herein are notinherently related to any particular computer or other apparatus. Inparticular, various general purpose machines may be used with programswritten in accordance with the teachings herein, or it may be moreconvenient to construct a more specialized apparatus to perform therequired method steps. The required structure for a variety of thesemachines will appear from the description given below.

In addition, embodiments of the present invention further relate tocomputer readable media or computer program products that includeprogram instructions and/or data (including data structures) forperforming various computer-implemented operations. The media andprogram instructions may be those specially designed and constructed forthe purposes of the present invention, or they may be of the kind wellknown and available to those having skill in the computer software arts.Examples of computer-readable media include, but are not limited to,magnetic media such as hard disks, floppy disks, and magnetic tape;optical media such as CD-ROM disks; magneto-optical media such asfloptical disks; and hardware devices that are specially configured tostore and perform program instructions, such as read-only memory devices(ROM) and random access memory (RAM). Examples of program instructionsinclude both machine code, such as produced by a compiler, and filescontaining higher level code that may be executed by the computer usingan interpreter.

FIG. 2 illustrates a typical computer system in accordance with anembodiment of the present invention. The computer system 100 includesany number of processors 102 (also referred to as central processingunits, or CPUs) that are coupled to storage devices including primarystorage 106 (typically a random access memory, or RAM), primary storage104 (typically a read only memory, or ROM). As is well known in the art,primary storage 104 acts to transfer data and instructionsuni-directionally to the CPU and primary storage 106 is used typicallyto transfer data and instructions in a bi-directional manner Both ofthese primary storage devices may include any suitable computer-readablemedia such as those described above. A mass storage device 108 is alsocoupled bi-directionally to CPU 102 and provides additional data storagecapacity and may include any of the computer-readable media describedabove. Mass storage device 108 may be used to store programs, data andthe like and is typically a secondary storage medium such as a hard diskthat is slower than primary storage. It will be appreciated that theinformation retained within the mass storage device 108, may, inappropriate cases, be incorporated in standard fashion as part ofprimary storage 106 as virtual memory. A specific mass storage devicesuch as a CD-ROM 114 may also pass data uni-directionally to the CPU.

CPU 102 is also coupled to an interface 110 that includes one or moreinput/output devices such as such as video monitors, track balls, mice,keyboards, microphones, touch-sensitive displays, transducer cardreaders, magnetic or paper tape readers, tablets, styluses, voice orhandwriting recognizers, or other well-known input devices such as, ofcourse, other computers. Finally, CPU 102 optionally may be coupled to acomputer or telecommunications network using a network connection asshown generally at 112. With such a network connection, it iscontemplated that the CPU might receive information from the network, ormight output information to the network in the course of performing theabove-described method steps. The above-described devices and materialswill be familiar to those of skill in the computer hardware and softwarearts.

The hardware elements described above may implement the instructions ofmultiple software modules for performing the operations of thisinvention. For example, instructions for matching selection vectors topatient electronic profiles (in electronic discharge records forexample) may be stored on mass storage device 108 or 114 and executed onCPU 108 in conjunction with primary memory 106.

C. PROCEDURES

FIG. 3 is a process flow diagram illustrating some of the importantsteps that may be employed in a preferred embodiment of the presentinvention. At least some of these steps are implemented as software ormachine operations on an appropriately configured computer system asdescribed above. These operations are preferably performed sequentially,in a continuous fashion, by the software and/or appropriately configuredmachine. It is, of course, possible that various steps described hereare performed by two or more separately operating pieces of software orappropriately configured machines.

As shown in FIG. 3, a process 300 begins at 302 and then in a step 304,the system receives data for two or more methods. As explained above andin more detail below, these methods will typically be assays or otherappropriate test procedure for which results (data) are generated.

After the appropriate data for the two or more methods has been input atstep 304, the system next arranges the data so that the results (e.g.,assay readings) are the dependent variables and the “true values” of thephysical property or other data reflected by the observed results arethe independent variables. Sometimes it will be necessary to estimatethe “true values” of the independent variables. Often, a first estimateof such independent variables is simply the value attributed to theindependent variable by the current calibration curve for the assay atissue. In FIG. 3, this step of arranging the dependent and independentvariables is performed by the system at a step 306.

Next, at an optional step 308, the system “cleans up” the data byremoving outliers and/or “noise” data which is clearly erroneous. Hence,the effects of outliers and erroneous results tend to be filtered out.This reduces the impact of atypical data on the performance of thetechnique. Outliers are the data points that most clearly fall outsideof the expected range of measured values. Usually, they are the resultof experimental error or other problem unrelated to the sample. Thus,step 308 serves to improve the quality of the nomograms generated by themethods of this invention.

As shown in FIG. 3, at a step 310, the forms of the response curveexpressions must be specified for each method. For example, the responsecurve may be specified as linear, quadratic, sigmoidal, etc. The form ofan expression for an assay is often provided by the vendor of that assayin the form of a calibration curve, for example. Sometimes, in order tosimplify the functional form of the curve and to simplify the noisedistribution over a dynamic range, logarithms of the dependent andindependent variables are used.

Once the form of the expression or expressions have been set, themathematical relationships between various parameters are given. Theseparameters and their corresponding relationships (as specified by themathematical expression) are now in a form that can be solved, given asufficiently complete set of data. As indicated at step 312, the systemreceives or specifies these expressions with the appropriate parametervalues. In addition, the expressions include one or more correctionfactors which are provided as unknowns. Thereafter, the unknowns (theparameters for the expressions and the correction factors for theindependent variables) are solved as a system of equations at a step314. These values are output and the process is completed at 316.

While process 300 is presented as a defined sequence of process steps.The invention is not limited to software and system that performs eachand every one of these steps. For example, it is not necessarily limitedto systems that remove “outliers” or automatically postulate the form ofthe expressions of the methods. Nor is the invention limited toprocesses which perform steps in the exact order specified by the flowchart of FIG. 3.

The data used in the methods of this invention is obtained from multiplesamples which are used in the various assays to generate measuredvalues. Generally, these samples are independently provided (e.g., fromvarious clinical samples and/or from specially made standards) so thattheir relative positions with respect to one another on the independentvariable axis are not accurately known. In some cases, some or all ofthe samples form a dilution series. That is, one sample, the originaland most concentrated sample, serves as the concentrate. Other samplesare created by diluting the concentrate with fixed amounts of thediluent. The relative positions of the dilution series members arerather clear, but the positioning accuracy is limited by the dilutionaccuracy. So some small correction between the relative positions of thedilution series samples may still be required, a single largerpercentage correction can be applied to the original and appropriatelyproportioned to each dilution member. In log space, this simply shiftsthe dilution sequence as an ensemble by the correction amount. Toimprove the quality of the input data, some samples may be measuredmultiple times with the same assay. These multiple readings will producea spread in the measured values.

The techniques of this invention require enough good data points(combinations of methods and samples) to completely “determine” thesystem. That is, the number of data points should at least equal thenumber of parameters and correction factors specified in 312 plus one.

As indicated at step 306, the method readings (results or observations)are provided as the dependent variables and the “true values” of thevariable being measured are provided as the independent variables. Forexample, in an assay for a particular biological compound or organism,the dependent variables may be readings of luminances, radioactivity,color, etc. Typically, the vendor will provide a calibration curve whichcorrelates these observed outputs with the independent variables. Asmentioned above, one is never completely sure that the “true values” ofthe independent variables are in fact accurate. Correcting the relationof these “true values” with multiple assays is a primary goal of thisinvention.

Referring now to FIG. 4, a hypothetical plot 400 presents an observedresult on the y-axis as a function of the true value on the x-axis fortwo hypothetical assays. A first response curve 402 represents theexpected behavior of a first assay (assay A) and a second response curve404 represents the expected behavior of a second assay (assay B). Notethat both curves 402 and 404 are fairly linear over a dynamic range(bounded by dashed vertical lines) but decrease in slope near themaximum and minimum regions of the dynamic range. Thus, the overallshape of curves 402 and 404 may be characterized as sigmoidal. Thisresponse is typical of many conventional assays.

Generally, the data will clearly show a trend indicative of the form ofa mathematical expression (e.g., linear). When the data is taken from awell characterized assay or other test, an appropriate form of themathematical expression may be provided in the literature. Curves 402and 404 are generally linear in a central region represented by theirdynamic ranges, although curve 404 is sufficiently curved that it willbe fit with a quadratic expression in this illustration.

Initial locations of the samples on the x-axis must be approximated.Some specially made samples will have “bottle values” (i.e., assignedapproximate concentrations). However, other samples such as clinicalsamples may have no preassumed property value. For these, fairlyreasonable initial property values must be provided. The normalizationtechnique of this invention will then correct the values through thecorrection factors on the x values. One suitable technique forapproximating the x value of unknown samples is to take the average ofthe measured results (y values) for the assays used to measure itweighted by sensitivity (steepness of slope). At first blush, this mayseem odd because the average gives a y value, not an x value. However,because the response curves are assumed to be monotonic, the order andrelative positions of the approximated x values for the samples will beroughly correct and random error is reduced by the averaging process.The x-axis will tend to represent assay readings and the measuredproperty indirectly. One can average the logarithm of the quantitatedproperty value across assays weighted by the sensitivity of each assay.Sensitivity is the steepness of the slope of logarithm assay readingsversus quantitated properties.

Before solving the set of expressions simultaneously, the anchor valuemust be chosen. This is the x value of one of the samples. Typically,the anchor value is chosen for the sample whose property value is mosttrusted. All x values of the other samples are defined relative to theanchor value. Therefore, all other sample x values on the axis arecalculated based upon their relations to the anchor value. To attainthis, correction factors are added to all property values on theindependent variable axis (the values for which samples were produced)except the one referenced as the anchor value.

Various mathematical expressions for the response curves may beemployed. While any monotonic non-constant function will work as anexpression for the response curves, simpler expressions are preferred asthey require fewer parameters. Each additional parameter removes adegree of freedom from the system. That is, additional data points arerequired to completely determine the system. Preferred expressions forresponse curves include linear expressions, quadratic expressions, andsigmoidal expressions.

If a data curve is postulated to be a line (step 310), then the form ofthe equation is

y=mx+b

with m representing the slope of the line and b representing the line'sintercept. These parameters must ultimately be solved for in step 314.

If the data curve is postulated to be a quadratic curve, the form of theequation is

y=kx ²⁺ mx+b

where the parameters are as defined above, and “k” is the quadratic termcoefficient. In this case, k, m, and b must be for in step 314.

If the data curve is postulated to be a symmetrical sigmoid, then it maybe represented by the expression

y=a+b/(1+exp(−cx+d))

In this expression, the slope of the sigmoid is bc/4 at x=d/c, and a, b,c, and d must be solved for in step 314. Asymmetrical sigmoids may bebetter than symmetrical sigmoids at fitting some data. Any modificationof the above expression which removes the symmetry of the derivative ofy with respect to x will introduce an asymmetry in the sigmoid; howeverthe derivative must be greater than zero for all x. Specific examples ofasymmetric sigmoids include those in which the function “−cx+d” in theabove expression is replaced with (−clnx+d), (−cx²+d) (for anynon-negative x), or any nonlinear polynomial constrained to satisfy theabove derivative requirements. Applications of asymmetric sigmoidfunctions are described in Chiu et al., “Use of Neural Networks toAnalyze Drug Combination Interactions,” American Statistical Assoc. 1993Proceedings of the Biopharmaceutical Section, pp 56-59 and J. Minor,“Neural Networks for Drug Interaction Analysis,” American StatisticalAssoc. 1993 Proceedings of the Biopharmaceutical Section, pp. 74-81.Both of these references are incorporated herein by reference in theirentireties and for all purposes.

Before the parameter values can be solved, the expression for eachmethod is written with a correction factor for the value of theindependent variable x (step 312). The quadratic form, for example, isthen written as

y=k(x1+δ1)² +m(x1+δ1)+b

where δ1 is the correction factor for the particular samplecorresponding to the current value of x(x1). For each sample used for aparticular method, the appropriate mathematical expression is providedwith the independent variable expressed as x+δ. Together theseexpressions are solved to yield the parameter values and the correctionfactors for the independent variables.

Given the data set presented in graph 400, the relevant expressions forsimultaneous solution are now presented. Note again that response curve402 (method A) is assumed to be linear over its dynamic range and theresponse curve 404 (method B) is assumed to be quadratic over itsdynamic range. Typically, these values are input as matrix coefficientsor are organized as such by the computer system.

yA0=m _(A)(x0+δ0)+b _(A)

yA1=m _(A)(x1)+b _(A)

yA2=m _(A)(x2+δ2)+b _(A)

yA3=m _(A)(x3+δ3)+b _(A)

yA4=m _(A)(x4 +δ4)+b _(A)

yB1=k(x1)² +m _(B)(x1)+b _(B)

yB2=k(x2+δ2)² +m _(B)(x2+δ2)+b _(B)

yB3=k(x3+δ3)² +m _(B)(x3+δ3)+b _(B)

yB4=k(x4+δ4)² +m _(B)(x4+δ4)+b _(B)

yB5=k(x5+δ5)² +m _(B)(x5+δ5)+b _(B)

Note that these expressions do not account for replicate measurementsfor given assay/sample combinations. For example, there are tworeplicates at x0 for assay A. In preferred embodiments, the systems ofthis invention generate a separate expression for each replicate.

Thus, there will actually be two expressions for sample x0 as evaluatedby assay A. These expressions will be identical, save the y value. Inthis example, rather than using the single expressionyA0=m_(A)(x0+δ0)+b_(A) for sample x0/assay A, the system would actuallygenerate two expressions:

yA0₁ =m _(A)(x0+δ0)+b _(A) and

yA0₂ =m _(A)(x0+δ0)+b _(A)

All expressions for all data points are then solved together (step 314)to yield the parameters m_(A), b_(A) (the slope and intercept of curve402), k, m_(B), and b_(B) (the curvature, slope and intercept of curve404) and correction values (δ0, δ2, δ3, δ4, and δ5). Note that x1 is theanchor value. Note also that this system is completely determined butnot over determined; there are 10 unknowns and 10 equations (one foreach data point). Often the system must be over determined; that is,there will be more expressions (data points) than unknowns in order toestimate precision (assay noise).

The unknown parameters may be solved by a non-linear regressiontechnique. Such techniques take as their objection functions the sum ofthe squares of the residuals (summed over all data points). Othersolution techniques such as maximum entropy, minimum entropy, andmaximum likelihood have other objective functions. In each case, thetechnique solves the unknowns by minimizing the objective function whichis a function of the unknown parameters that measures optimal fit to thedata. A computer technique performs the required minimization overmultiple iterations until the problem converges.

The residual used in the sum of squares of residuals objective functionis given by

R=Y _(j) −f(X _(j), δ_(i), β_(i))

where j identifies the data point under consideration, i identifies thecurrent iteration in an iterative technique to minimize the costfunction, Y is the actual measured value of data point j, f is thefunctional form of the response curve (e.g., quadratic), δ is thecorrection factor for the sample used in data point j, β is thestructural parameter (or parameters) used in the functional form (e.g.,k, m, and b for a quadratic), and R is the residual. The sum of squaresof residuals cost function is simply Σ(R_(j))² over all values of j(i.e., over all data points). Note that in conventional regressiontechniques, the values of X_(j) are assumed to be precise. In contrast,the methods and systems of this invention initially take the X_(j) asimprecise and then correct them to be precise.

The present invention preferably, though not necessarily, employs aMarquardt-Levenberg method, which is an iterative method to minimize thecost function, whatever it may be. D. W. Marquardt “An Algorithm forLeast-Squares Estimation of Nonlinear Parameters” Isoc. Indust. Appl.Math. (SIAM), Vol. II, No. 2, pp. 431-441; and Yonathan Bard, “NonlinearParametric Estimation,” Academic Press, New York, N.Y., 1974. Thesereferences are incorporated herein by reference in their entireties andfor all purposes. The Marquardt-Levenberg method stabilizesover-determined systems during minimization of the cost function. Therelevant matrix equation for minimizing the sum of squares of residualsmay be represented as follows: ${\begin{bmatrix}{\frac{\partial f_{i}}{\partial\delta_{i}},{\ldots \quad 0},\frac{\partial f_{i}}{\partial\delta_{m}},{\ldots \quad \frac{\partial f_{i}}{\partial\beta_{i}}},{\ldots \quad \frac{\partial f_{i}}{\partial\beta_{l}}},\ldots} \\\vdots \\{\frac{\partial f_{i}}{\partial\delta_{i}},{\ldots \frac{\partial f_{i}}{\partial\delta_{l}}},0,{\ldots \quad \frac{\partial f_{i}}{\partial\beta_{i}}},{\ldots \quad 0},\frac{\partial f_{i}}{\partial\beta_{m}},\ldots} \\\vdots\end{bmatrix}\begin{bmatrix}{\Delta\delta}_{i} \\\vdots \\{\Delta\delta}_{n} \\{\Delta\beta}_{i} \\\vdots \\{\Delta\beta}_{p}\end{bmatrix}} = \begin{bmatrix}r_{i} \\\vdots \\r_{i} \\\vdots\end{bmatrix}$

Here the matrix on the left hand side of the equation is the gradientmatrix (sometimes referred to as a Jacobian) of the response curvefunctions. Each row corresponds to a different data point j, and eachcolumn corresponds to a different unknown structural parameter, β, orcorrection value, δ, to the sample value. The individual elements of thegradient matrix are the first derivatives of the response curvefunctions with respect to either a structural parameter or a correctionvalue. For example, the element in the first column of the first row(j=1) might be the derivative of the response curve expression (e.g., aquadratic expression) for the method used to analyze the first datapoint with respect to the curvature, k, for that expression. Thederivative is evaluated for the current values of the parameters andcorrection factors appearing therein; that is, the parameters andcorrection factors of the current iteration. The element in the secondcolumn in the first row of the gradient matrix will be the derivative ofthe same response curve expression but with respect to a differentparameter or correction value—possibly the correction factor δ1. Theremainder of the first row is populated with derivatives of theexpression at the first data point with respect to each unknownparameter and correction value in the entire system. The second row issimilarly populated, but this time with derivatives of the function atthe second data point; i.e., the response curve of the method used toanalyze the second data point and evaluated at the second data point.There is one row for each data point in the gradient matrix. And thereis a column for each unknown parameter and correction value.

The residual vector, R_(j), on the right hand side of the matrixequation is just the value of the residual, R, for each data row. Thus,there is a row for each data point in the data set. The residual valuesare calculated per the above equation for R.

It should be understood that each and every data point may berepresented as a row in the gradient matrix (and an entry in theresidual matrix). This includes replicate measurements taken for asingle sample/assay combination. As mentioned, there will be some spreador intrinsic noise (typically a Gaussian) in the measured values takenfor each sample/assay combination. The technique described hereinhandles such noise considering each of the replicate measurements takenand provides the best curve fit through them on the best representationof x-axis spacing between samples.

As shown, the gradient matrix is multiplied by a vector delta values.These are the changes—between iterations—for each unknown structuralparameter and correction value. The number of rows in this vector equalsthe total number of unknown structural parameters and correction values.Thus, the number of rows in the delta vector equals the number ofcolumns in the gradient matrix. The delta vector is the unknown quantitywhich is solved for in each iteration. The delta values of theparameters are added to the values of those parameters from the previousiteration to obtain the improved values of the parameters for the nextiteration.

To solve this matrix equation for the delta parameter values, a modified“least squares projector” may be employed. Using this technique, thedelta vector is solved according to the following equation:$\begin{bmatrix}{\Delta \overset{\rightharpoonup}{\delta}} \\{\Delta \overset{\rightharpoonup}{\beta}}\end{bmatrix} = {{\left\{ {\left\lbrack {\begin{bmatrix}{\nabla f} \\\left( {\overset{\rightharpoonup}{\delta},\overset{\rightharpoonup}{\beta}} \right)\end{bmatrix}^{T}\begin{bmatrix}{\nabla f} \\\left( {\overset{\rightharpoonup}{\delta},\overset{\rightharpoonup}{\beta}} \right)\end{bmatrix}} \right\rbrack + \lambda} \right\}^{- 1}\begin{bmatrix}{\nabla f} \\\left( {\overset{\rightharpoonup}{\delta},\overset{\rightharpoonup}{\beta}} \right)\end{bmatrix}}^{T}\lbrack R\rbrack}$

In a normal least squares projector, the gradient matrix is multipliedby its transpose and the inverse of resulting square matrix is taken.This inverse is then multiplied by the transpose of the gradient matrixand that result is multiplied by the residual vector. This approach willgive the delta vector, but it may be unstable due to singularities(i.e., the system may divide by zero during the operation of the leastsquares projector without some modification).

To address this instability, the Marquardt-Levenberg approach introducesa scalar factor, λ>0, as indicated in the above expression.Specifically, λ is added to the product of the gradient matrix and itstranspose before the resulting square matrix is inverted. By adding thescalar λ, the system avoids singularities. An intial value, λ₀, isguessed and then adjusted (typically by trial and error) until the valueof the objective function is reduced. Generally, the value of λ₀ ischosen to be as small as possible (about 0.01 in one specific example).The smaller the value of λ, the faster the overall problem converges toa solution. However, for each iteration, the value of λ must besufficiently large that the system is stable and that the sum of thesquares of the residuals is actually reduced in comparison to theprevious iteration.

With the delta values now in hand, the parameters and correction factorsfor the next iteration (iteration i+1) are calculated as follows:

δ_(i+1)=δ_(i)+Δδ_(i)

β_(i+1)=β_(i)+Δβ_(i)

At this point, the objective function (sum of squared residuals in thisexample) is evaluated.

This may be accomplished by putting the values of δ_(i+1) and β_(i+1)back into the expressions for the residual vector and then multiplyingthat newly evaluated residual vector by its transpose. Then the computersystem determines whether the current value of the objective function isless than the previous value of that function. If the objective functionis in fact less than in the previous iteration, the value of λ is madesmaller. The new value of the objective function is calculated. If it isstill smaller, then λ is again reduced. This continues until the valuethe objective function increases over that of the previous iteration.All this takes place for a given iteration. If the initial value of λgenerates a objective function that is greater than that of the previousiteration, then the value of λ is increased until the objective functionfalls below that of the previous iteration.

After each such iteration, the new parameter values are put in the abovematrix equation to generate new delta vectors until some criterion forstopping is reached. Generally, suitable criteria require that the deltavector or the objective function becomes quite small. For example, theiterations may cease when the one of these reaches zero. Sometimes, thecorrection vector or objective function will not quickly reach zero. Toaccount for such cases, a stopping criterion may be a suitably smallvalue of these over multiple iterations.

When the stopping criterion is reached, all structural parameters forthe response curves are known as are the correction factors to the trueproperty values of the samples. The computer system then outputs thesesolved values. The outputs are then provided as expressions for theresponse curves of each method as a function of a common independentvariable (the true property values of the samples). These expressionsmay be plotted together to allow visual comparison of theirresponsiveness, sensitivity, linearity, etc.

In one example, the resulting nomogram may be used to convert values ofone assay to the equivalent value obtained by another assay according tothe following four steps procedure.

1. Locate the assay value to be converted on the left side scale of theplot (the y-axis).

2. Move right, parallel to the x-axis until that assay's curve islocated.

3. Next, move up or down parallel to the y-axis until the curve of theassay whose comparative value is desired.

4. Finally, move parallel to the x-axis back to the assay scale to readthe expected value of the comparative assay.

While the above discussion has presented a two dimensional problem(measured y-values versus underlying or true property x-values), thereis in principle no reason why the computer techniques of this inventioncan not be applied to systems having additional dimensions (axes).

Those additional dimensions or axes represent other independentvariables (x values) manifesting the same underlying condition or stateof the physical system. For example, the disease hepatitis B representsan underlying condition having multiple manifestations, each of whichprovides a separate independent variable (x-axis). These manifestationsmay be (1) concentration of HBV DNA, (2) concentration of HBV surfaceantigen (HBSAg), (3) concentration of immunoglobulins to HBV, etc.Various assays measure these various manifestations of hepatitis Bdisease. If clinical samples from heptitis B patients are tested withthese various assays, the results may be normalized with a method of thetype described above. For example, a sum of least squares objectivefunction may be minimized by an iterative technique such as theabove-described Marquardt-Levenburg technique. However, the data will beprovided for two more independent variables. Thus, correction factorsfor the samples to be corrected are provided for each of the independentvariable axes. Such techniques allow normalization of data acrossmultiple independent axes to permit greater insight into the underlyingphenomenon (e.g., hepatitis B disease).

D. APPLICATIONS

Various applications for the techniques of this invention are possible.One important application is conversion among results from differentassays. This allows physicians or others working with multiplemeasurement techniques to use data from all techniques with confidencethat the data can be interpreted consistently. For example, an HIVpatient may periodically receive a blood analysis in which HIV viralload is monitored by a first method. However, one reading is made by asecond method (which quantitates by a fundamentally different mechanismthan the first method). Using the conversions afforded by thisinvention, a physician interpreting the data from both methods can beconfident that his reading of the second assay results provide an HIVviral load that is consistent with the readings from the first assay.

The multi-measurement technique also allows construction of models thatdraw on data from two more different assays. Before this invention, whenno reliable method for converting between results of multiple assays wasavailable, one was limited to models that employed data from a singleassay. Now the combined results from multiple assays can generate themodel, validate the model, and/or provide inputs to execute the model.

The multi-measurement technique of this invention can also compare theperformance of the multiple methods that it compares. Prior to thisinvention, it was impossible to compare three or more distinctmeasurement methods by a single regression analysis. Traditionalregression analysis is also unable to distinguish the error rate or theintrinsic bias of the evaluated assays.

Aspects of assay performance are contained within the equation ofquadratic assay curves which are written in the form: y=kx²+mx+b. Thecurvature (k) provides an indication of the linearity of the assay; thelower the curvature (k approaches 0), the more linear the assay. Theslope (m) is an average measure of the responsiveness of the assaythroughout its dynamic range, in other words, how measured analytequantification values change in response to the approximated “true”analyte concentration (i.e, the sensitivity at x). The slope at anyparticular point along the curve (the instantaneous slope) may bederived from the equation: dy/dx=2kx+m. The greater the instantaneousslope, the greater the responsiveness of the assay at any given pointwithin the dynamic range of the assay. Finally, the precision of eachassay is indicated by the amount of scatter which is represented by theR² value, a measure of how closely the assay curve fits the data set. Ahigher number indicates less scatter and therefore a greater degree ofprecision. Optionally, the standard deviation (estimated noise spreadabout the curve) is a suitable measure of precision for each assay.

Another application for the multi-measurement technique is to assess anddevelop standards for new assays. Standards give assays a commonreference so that relative comparisons have meaning. Typically, an assayis calibrated against a standard which supposedly represents some fixedvalue of the independent variable (e.g., analyte concentration). Thispresents two problems. First, some standards, particularly those usedfor biological assays, can degrade or “float” with time. In such cases,the standard moves over time. Second, when a standard is used up, anenterprise must produce a new batch of standard. Unfortunately, it canbe very difficult to produce fresh standards which accurately reproducethe preceding standard(s). It is not uncommon for assigned (but unknown)property values for bioassay standards to be off by 20 to 30 percent.Thus, the new standard must always be calibrated against the old. Thisinvolves the same conversion problems discussed above with respect tomultiple assays.

Since it leverages the relative spacing of x-values of samples, themultiple measurement techniques of this invention can be used to assistin generating new standards and comparing existing standards. First,biochemists (or other technical personnel) making the new batches ofstandards can check the putative standards against previous standardsusing the multi-measurement technique to determine whether the newstandard is sufficiently close to the old. If not, the new standard canbe adjusted or reformulated until it reaches a level sufficiently closeto the old standard. Second, the multi-measurement technique can exactlyquantify any separation between the new and old standards. Thisinformation is then used to modify the calibration curves or softwareprovided with assays employing the new standard. Still further, themulti-measurement technique can check the quality of assays going outthe door (i.e., quality control).

Like any other application of the inventive technique, more than onemeasurement method must be used to analyze the standard. Obviously,distinct methods could be used. Often, however, it would be moreconvenient to use a single method (e.g., one company's assay for HBV).In such cases, different “versions” of the same assay can be used. Notethe definition of “different” methods can be quite general, e.g. thesame assay under “different” circumstances such as different plates,different calibrator batches, or different types of calibrator. Sincethe methods are measuring the same samples, inherent in theirmeasurements should be the essence of a common target (communality) butfrom different “angles.” In effect, no one method produces the correctanswer, but assuming reasonable distribution about the true value, theensemble result from all methods is expected to be more accurate. Hence,a self-consistent approach provides excellent information on this commonbut unknown target (the new standard). Each method produces a “y” value(typically logarithm of the method reading), representative of theanalyte level in the standard. Since an assay “A” is judged worldwide byother assays, using those other assays with assay “A,” to leveragequantiation standards for assay “A” assures the best possible worldwiderating for assay “A.”

E. EXAMPLES

Specimens (247) from patients with chronic hepatitis B were evaluatedfor Hepatitis B virus (HBV) DNA using three commercial assays—ChironQuantiplex™ (CA), Digene Hybrid Capture (DA) and Abbott HBV DNA assay(AA). The results were used to generate a multi-measurement techniquenomogram that compared the assay's linearity, responsiveness andprecision, and allowed conversion between the different assay values.The multi-measurement technique analysis showed the CA to be moresensitive and responsive than the DA and AA. Both CA and DA were moreprecise than AA. Cross validation of the multi-measurement techniqueresults was performed using two additional data sets of 500 and 200paired specimens, respectively.

Clinical Specimens

A total of 247 serum specimens were obtained from 76 chronic hepatitis Bpatients. 113 of these specimens were sequential specimens obtained frompatients receiving interferon therapy as part of a interferonre-treatment trial. All specimens were originally tested for HBV DNA bythe CA. Subsequently, subsets of specimens were tested by the DA and AA.In all, 113 were independently assayed by all three HBV DNAquantification assays, 114 by the CA and AA only, and 20 by the CA andDA only. All sera were separated within four hours of collection andeither immediately frozen at −70° C. or kept at 4° C. for 24-28 hoursprior to freezing at −70° C. Sera underwent only one freeze-thaw cycleprior to HBV DNA testing.

HBV DNA Quantification

All three assays were performed according to the instructions of theirmanufacturers. CA (Chiron Corporation, Emeryville, Calif.) uses a seriesof synthetic oligonucleotide probes to bind single-stranded HBV DNAmolecules to a solid phase and uses branched DNA (bDNA) technology togenerate a chemiluminescent signal which corresponds to the amount ofHBV DNA target in the specimen. For the CA, all specimens were tested induplicate, and the concentration of HBV DNA in each specimen wasdetermined from a standard curve. DA (Digene Diagnostics, Inc.,Beltsville, Md.) uses HBV-RNA probes to capture single stranded HBV DNAmolecules to a solid phase, and anti-RNA:DNA hybrid antibodiesconjugated to alkaline phosphates to generate a chemiluminescent signal.For the DA, specimens were tested individually and the concentration ofHBV DNA in each specimen was determined by comparison to thechemiluminescent signals derived from controls which were tested induplicate. AA (Abbott Laboratories, Abbott Park, Ill.) is a liquidhybridization assay in which HBV DNA is hybridized to single-stranded¹²⁵I-HBV DNA probes and unhybridized DNA is removed by sepharosechromatography. For the AA, specimens were tested individually and theconcentration of HBV DNA in each specimen was determined by comparisonto the radioactivity of positive and negative controls.

Comparison of Assay Performance

Of the three assays, the CA exhibited better performance with regards tolinearity, responsiveness and precision. The k value (curvature), ameasure of assay linearity, for the cA was lower than the k values forthe DA or AA (0.15 versus 0.61 and 0.81, respectively). Both CA and DAhad less scatter, as indicated by higher R² values, than the AA (0.9951and 0.9946 versus 0.9614, respectively).

Conversion between Assay Values

The multi-measurement technique nomogram also provides a means toconvert between the different assay values. Conversion of assay valuesmay be accomplished by using the assay curves shown in FIG. 5 to compare“y” values between assays for a given “x” value, or by using theequations shown in FIG. 6. Examples showing the conversion of a CA valueof 100 Meq/ml to a DA value of 150 pg/ml using both the assay curves andthe equations are described below.

Using the assay curves to convert a CA value of 100 Meq/ml to a DA valuein pg/ml one would: 1) locate 100 on the y-axis, 2) move horizontallyaway from the y-axis to the CA curve, 3) move vertically up to the DAcurve, and 4) move horizontally toward the y-axis to read 150.

Using the equations to convert a CA value of 100 Meq/ml to a DA value inpg/ml one would: 1) set y equal to 100 and solve the equation“y=0.15x²+3.79x+4.96 to obtain a value for x, and 2) use this x-value tosolve the equation “y=0.61x²+3.34x+5.21” and obtain a value of 150 fory.

It is important to recognize that, although there is overlap between thedynamic range of the different assays, specimens whose values fallbeyond the dynamic range of a given assay (e.g. DA values >2000 pg/ml)must be diluted in HBV DNA negative serum and be re-tested to obtain anaccurate viral load measurement. Similarly, the AA is unable toaccurately quantify specimens containing <4-5 pg/ml.

Cross Validation of Multi-Measurement Method

The ability of the multi-measurement technique to accurately convertbetween assay values was tested in two ways: (1) by comparing themulti-measurement technique-derived relationships with previouslypublished relationships, and (2) by comparing the multi-measurementtechnique-derived quantification values with measured values from anindependent data set. For the first analysis, the multi-measurementtechnique-derived relationship between the CA and AA was compared to apreviously-published relationship between the CA and AA that was derivedusing linear regression techniques (Kapke, G. F., G. Watson, S.Sheffler, D. Hunt, and C. Frederick. 1997 “Comparison of the ChironQuantiplex branched DNA (bDNA) assay and the Abbott Genostics solutionhybridization assay for quantification of hepatitis B viral DNA” J ViralHepatitis. 4:67-75). The correlation coefficient between themulti-measurement technique and linear regression curves was 0.995 forHBV DNA quantification values above 100 Meq. The close correlationbetween these CA and AA relationships is likely because the more preciseCA values were plotted on the x-axis for the linear regression analysis,thereby providing a greater degree of agreement between the linearregression and multi-measurement technique curves.

The ability of the multi-measurement technique to accurately convertbetween DA and CA values was assessed by testing a separate cohort of100 chronic hepatitis B patients by both assays. The measured DA valuesthen were compared with the values predicated by the multi-measurementtechnique based on the CA values. There was a close correlation betweenthe measured DA values (observed HBV DNA) and the multi-measurementtechnique-predicated DA values (expected HBV DNA). The correlationcoefficient (R²) for this analysis was 0.96.

The multi-measurement technique-derived correlations are consistent withprevious observations. The greater linearity of the CA as compared tothe DA and AA has been noted in a recent study by Butterworth andcolleagues (Butterworth, L. -A., S. L. Prior, P. J. Buda, J. L.Faoagali, and G. Cooksley. 1996 “Comparison of four methods forquantitative measurement of hepatitis B viral DNA” J Hepatol.24:686-691). With regard to precision, the CA consistently has beenreported to exhibit the best performance, although the relativeperformance of the DA and AA varies (Butterworth, L. -A., S. L. Prior,P. J. Buda, J. L. Faoagali, and G. Cooksley 1996 “Comparison of fourmethods for quantitative measurement of hepatitis B viral DNA” JHepatol. 24:686-691; Kapke, G. F., G. Watson, S. Sheffler, D. Hunt, andC. Frederick. 1997 “Comparison of the Chiron Quantiplex branched DNA(bDNA) assay and the Abbott Genostics solution hybridization assay forquantification of hepatitis B viral DNA” J Viral Hepatitis 4:67-75). Thegreater sensitivity (i.e. greater responsiveness at the lower end of thedynamic range) of the CA over the DA and/or AA also has been noted inseveral studies (Zaaijer, H. L., F. ter Borg, H. T. M. Cuypers, M. C. A.H. Hermus, and P. N. Lelie. 1994 “Comparison of methods for detection ofhepatitis B virus DNA” J Clin Microbiol. 32:2088-2091; Butterworth, L.-A., S. L. Prior, P. J. Buda, J. L. Faoagali, and G. Cooksley. 1996“Comparison of four methods for quantitative measurement of hepatitis Bviral DNA” J Hepatol. 24:686-691; Kapke, G. F., G. Watson, S. Sheffler,D. Hunt, and C. Frederick. 1997 “Comparison of the Chiron Quantiplexbranched DNA (bDNA) assay and the Abbott Genostics solutionhybridization assay for quantification of hepatitis B viral DNA” J ViralHepatitis. 4:67-75; Pawlotsky, J. M., A. Bastic, I. Lonjon, J. Remire,F. Darthuy, C. J. Soussy, and D. Dhumeaux 1997 “What technique should beused for routine detection and quantification of HBV DNA in clinicalsamples?” J Virol Methods. 65:245-253).

Limited Overlap Example

The multiple measurement technique of this invention was used to assessthe performance characteristics of the Roche AMPLICOR HBV Monitor PCRTest (RA) and its relationship to the Chiron Quantiplex® HBV DNA assay(CA), Digene Hybrid Capture™ (DA), and Abbott HBV DNA assays (AA). Wetested in duplicate sixty specimens previously evaluated by theabove-described HBV DNA assays by the RA and generated a newmulti-measurement technique graph incorporating the results from all 4assays. The curves generated by the multi-measurement technique methoddemonstrate excellent concordance between the CA and the RA in the smallarea of overlap (R²=0.99). They also define the region of overlap of theRA and CA to be 0.7-8 MegaEquivalents/mL (MEq/mL)=700,000-8,000,000copies/mL (or 2.5-28.2 pg/ml) based on the high end plateau of the RA. Apreliminary sensitivity analysis using clinical specimens conservativelyapproximates the minimum detection limit of the RA—defined as the valueabove which specimens can be considered positive with 95% certainty(approx. 3500 copies/ml). In addition, the reliable detection limit ofthe RA—the value at which one can reliably quantify specimens with 95%confidence—appears to be higher. Based on our data, the RA appearsapproximately 3 logs more sensitive than the CA and DA. Of 512 specimenstested for HBV DNA, 268/512 (52%) quantify as <0.7 MEq/mL, 52/512 (10%)quantify between >0.7 and <8 MEq/mL and 192/512 (38%) quantify as >8MEq/mL. Thus given the upper limit of quantification of the RA which isestimated to be approx. 8 MEq/mL, 38% of next specimens tested for HBVDNA will be beyond the dynamic range of the RA.

To facilitate comparison of assay performance and coversion betweenassay values for those assays that share a common dynamic range, anomogram was generated with contains response curves for the CA, DA, andRA assays. See FIG. 7A. The DA results (circles) are fit to a curve 701,the CA results (Xs) are fit to a curve 703, and the RA results(triangles) are fit to a curve 705. The assay results were plotted on alogarithm transformed scale, where the y-axis indicates measured HBV DNAlevels in the specimens measured by each of the three assays (in thevarious assay units), and the x-axis indicates the approximated “true”HBV DNA in each specimen as generated by the technique of thisinvention.

In this analysis, each assay's response curve was indicated by a fourparameter sigmoid curve: minimum, maximum minus minimum, slope, and the50 percent point. These parameters are illustrated in FIG. 7B.

H. ADDITIONAL EMBODIMENTS

While this invention has been described in terms of several preferredembodiments, it is contemplated that alternatives, modifications,permutations and equivalents thereof will become apparent to thoseskilled in the art upon a reading of the specification and study of thedrawings. It is therefore intended that the following appended claimsinclude all such alternatives, modifications, permutations andequivalents as fall within the true spirit and scope of the presentinvention.

What is claimed is:
 1. A computer implemented method of normalizing at least two measurement methods, the method comprising: receiving data specifying measured values versus physical properties for samples evaluated by the measurement methods, wherein the data includes (i) at least two first sample data points which were obtained from a first sample analyzed with at least two of the measurement methods and (ii) at least two second sample data points which were obtained from a second sample analyzed with at least two of the measurement methods; assuming a value for the physical property of the first sample and assuming a value for the physical property of the second sample; for each of the measurement methods, assuming a mathematical form of a response curve describing the measured values as a function of the physical properties, wherein the forms of expression contain unknown parameter values; and simultaneously solving for all the unknown parameter values and a correction factor for the value of the physical property of the second sample.
 2. The method of claim 1, further comprising providing a nomogram including solved response curves or solved mathematical expressions for at least two of the measurement methods.
 3. The method of claim 1, wherein the data received includes at least enough data points to fully determine the parameters values of the response curve, the correction factor for the second sample, and any additional correction factors.
 4. The method of claim 1, wherein the measurement methods are assays.
 5. The method of claim 1, wherein the measurement methods are assays for biological materials.
 6. The method of claim 1, wherein the measured values include at least one of luminescence, radioactivity, and color intensity.
 7. The method of claim 1, further comprising setting the value for the physical property of the first sample as an anchor value, wherein during simultaneously solving for all the unknown parameter values the anchor value is not provided with a correction value.
 8. The method of claim 1, wherein the mathematical forms of the response curves include at least one of linear expressions, quadratic expressions, and sigmoidal expressions.
 9. The method of claim 1, wherein the method simultaneously solves for one or more additional correction factors associated with physical quantities of additional samples for which data was received.
 10. A computer program product for normalizing response curves of plurality of measurement methods, the computer program product comprising: (a) a computer readable medium; and (b) instructions, stored on the computer readable medium, for normalizing the response curves, the instructions comprising (i) receiving data specifying measured values versus physical properties for samples evaluated by the measurement methods, wherein the data includes (i) at least two first sample data points which were obtained from a first sample analyzed with at least two of the measurement methods and (ii) at least two second sample data points which were obtained from a second sample analyzed with at least two of the measurement methods; (ii) assuming a value for the physical property of the first sample and assuming a value for the physical property of the second sample; (iii) for each of the measurement methods, assuming a mathematical form of a response curve describing the measured values as a function of the physical properties, wherein the forms of expression contain unknown parameter values; and (iv) simultaneously solving for all the unknown parameter values and a correction factor for the value of the physical property of the second sample.
 11. The computer program product of claim 10, where the instructions further comprise providing a nomogram including solved response curves or solved mathematical expressions for at least two of the measurement methods.
 12. The computer program product of claim 10, wherein the data received includes at least enough data points to fully determine the parameters values of the response curve, the correction factor for the second sample, and any additional correction factors.
 13. The computer program product of claim 10, wherein the measurement methods are assays.
 14. The computer program product of claim 10, wherein the measurement methods are assays for biological materials.
 15. The computer program product of claim 10, wherein the measured values include at least one of luminescence, radioactivity, and color intensity.
 16. The computer program product of claim 10, wherein the instructions further comprise setting the value for the physical property of the first sample as an anchor value, wherein during simultaneously solving for all the unknown parameter values the anchor value is not provided with a correction value.
 17. The computer program product of claim 10, wherein the mathematical forms of the response curves include at least one of linear expressions, quadratic expressions, and sigmoidal expressions.
 18. The computer program product of claim 10, wherein the instruction (iv) simultaneously solves for one or more additional correction factors associated with physical quantities of additional samples for which data was received.
 19. A computer implemented method of normalizing at least two measurement methods, the method comprising: receiving data specifying measured values versus physical properties for samples evaluated by the measurement methods; using the data to simultaneously solve for (i) structural parameters defining the shape of response curves for each assay, and (ii) one or more correction factors adjusting the relative positions of physical property values assumed for the samples; and outputting expressions for the response curves of each assay normalized on a common axis representing the physical property values.
 20. The method of claim 19, wherein simultaneously solving for structural parameters and correction factors involves mininimizing an objective function.
 21. The method of claim 20, wherein simultaneously solving for structural parameters and correction factors employs a regression technique which minimizes the sum of the squares of the residuals of the data points.
 22. The method of claim 20, wherein simultaneously solving for structural parameters and correction factors employs a Marquardt-Levenberg technique.
 23. The method of claim 19, wherein the received data includes replicate measurement values of a given sample by a given assay.
 24. The method of claim 19, wherein simultaneously solving for structural parameters and correction factors involves generating response curve expressions for each received data point and iteratively correcting the values of structural parameters and correction factors until convergence.
 25. The method of claim 19, further comprising creating a nomogram from the output normalized expressions for the response curves. 